Eric Riemer Hare
Eric Hare
Iowa State University
April 5th, 2017
Statistical Methods for Bullet Matching
Chapters:
The problems culminated in a 2009 NAS report which found “much forensic evidence – including, for example, bite marks and firearm and toolmark identification is introduced in criminal trials without any meaningful scientific validation, determination of error rates, or reliability testing.” (National Research Council 2009)
From a September 2016 report by the President’s Council of Advisors on Science and Technology (PCAST) titled Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods (Advisors on Science and Technology 2016):
A second—and more important—direction is (as with latent print analysis)
to convert firearms analysis from a subjective method to an objective
method. This would involve developing and testing image-analysis
algorithms for comparing the similarity of tool marks on bullets. [...]
In a recent study, researchers used images from an earlier study to
develop a computer-assisted approach to match bullets that minimizes
human input [338].
338: Hare, E., Hofmann, H., and A. Carriquiry. “Automatic matching of bullet lands.” Unpublished paper, available at: arxiv.org/pdf/1601.05788v2.pdf.
The key to this approach is the reference database…
plot3D.x3p.file(read_x3p("../images/Hamby (2009) Barrel/
bullets/Br1 Bullet 1-5.x3p"),
plot.type = "surface")https://isu-csafe.stat.iastate.edu/shiny/bulletr/
We need to choose a location (height) of the bullet at which to extract a profile. To do so, we optimize the CCF (T. Vorburger et al. 2011):
Parameters: \(d = 25\mu m, d_0 = 25\mu m, c = 0.9\)
br111 <- get_crosscut("images/Br1 Bullet 1-5.x3p", x = 243.75)
qplot(y, value, data = br111) + theme_bw()The striations that identify a bullet to a gun barrel are located in the land impression areas (Xie et al. 2009).
Parameters: \(s = 35\mu m\)
br111.groove <- get_grooves(br111)
br111.groove$plotbr116 <- "../images/Hamby (2009) Barrel/bullets/Br1 Bullet 1-6.x3p"
result2 <- get_grooves(get_crosscut(br116))
result2$plotLocal weighted scatterplot smoothing (Cleveland 1979) - Fits a low-degree polynomial to a small subset of the data, weighting values near the point to be estimated more strongly.
br111.loess <- fit_loess(br111, br111.groove)
br111.loess$fittedDeviations from the loess fit should represent the imperfections (striations) on the bullet. Hence, we extract the residuals from the model.
br111.loess$residAs with detecting the shoulders, we can smooth the deviations and compute derivatives to identify peaks and valleys in the signature.
br111.peaks <- get_peaks(br111.loess$data)
br111.peaks$plotThe previous five steps are performed for each bullet land. But now we wish to extract features for cross comparisons of bullet lands.
Features are extracted from each land-to-land comparison:
To begin to tackle the degraded bullet problem, we need to standardize features by the length of the recovered land.
Matches = 27, Matches per mm = 14.72
By standardizing the features, we don’t penalize the degraded case as in the first revision of our algorithm:
Matches = 8, Matches per mm = 11.42
Our algorithm had trouble in scenarios where large deformations in the two aligned signatures yielded a high CCF, when in fact the two land were not matches.
We fit a new loess fit to the average of the two signatures, and subtract the resulting fit from the original signatures - This models the “roughness” of the bullet land while removing the “waviness”
Simulation Study:
We simulated the degradation of the processed signatures rather than simulating a degraded surface.
This means that the recovered signature may have been slightly influenced by points that were removed in the simulation, though this effect is likely to be minor.
This confirms a NIST theory that the leading shoulder of the bullet better expresses striations as it travels through the barrel. To further test this, we can attempt to match a previously excluded land (due to severe tank rash) to its known match.
Extracting the ideal signature and then simulating a left-fixed 50% degradation scenario yields the following:
Verbatim from September:
We have taken steps to address each of these concerns…
NIST has provided some more data:
With two studies, Hamby 252 and Hamby 44, there are three sets of cross comparisons we perform:
Given that the barrels are the same in each case, assuming perfect scans and no microscope operator effects, each set of comparisons should be indistinguishable from one another.
In a real world forensic application of these algorithms, the true experimental unit is the bullet rather than the land
Idea: Two bullets \(b_1\) and \(b_2\) are matches if and only if some land \(l_{b1}\) from \(b_1\) matches some land \(l_{b2}\) from bullet 2. Corollary: Two bullets are non-matches if and only if no land from \(b_1\) matches a land from \(b_2\). Assumptions: Each land on a bullet is independent of each other land
The true alignment of the lands exists along one of 6 diagonals:
Therefore, we can multiply the probabilities across the diagonals and take the highest for each bullet to bullet comparison (Sensorfar 2017)
Assuming bullets match if and only if all lands match:
Performance is weaker in this case because our algorithm is much more likely to make false negatives compared to false positives.
Average the scores along the maximal diagonal
We once again achieve much clearer separation.
In Designing Modular Software: A Case Study in Introductory Statistics, we outlined several software design principles:
To truly open these bullet matching methods to the scientific community, we had to focus on a system which adheres to these principles
The underlying technology we’ve created since September is a brand new database which modularizes the algorithm and allows collaboration on individual components. The database:
https://isu-csafe.stat.iastate.edu/shiny/bullets/
By far the biggest limitation of these algorithms thus far is the limited amount of available 3D scan data for bullets:
11 unique gun barrels is not yet enough to form a true reference distribution for known matches and non-matches…
…However, the structure of the database means that as soon as new data is available, the features and scores can be easily recomputed.
Some work has been done to optimize the parameter choices (smoothing factor, optimal cross-section, etc.) But there are number of parameters which were chosen without cross-validation:
Further, the parameters we did optimize were chosen to be globally optimal rather than for each individual land or cross-comparison of lands. This was done for computational efficiency.
While we showed the presence of operator effects, we didn’t provide many recommendations for addressing this. Some possible solutions:
As Iowa State University and NIST continue to collaborate, we will likely have several more individuals scanning and uploading surface scans of bullets, and this will need to be further investigated to ensure the algorithms withstand the real-world scenario of a wide variety of scan quality.
Special thanks to Alan Zheng at the National Institute of Standards and Technology for maintaining the NIST Ballistics Toolmark Research Database and providing many useful suggestions for our algorithm.
Any Questions?
Advisors on Science, President’s Council of, and Technology. 2016. “Report on Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods.” https://www.whitehouse.gov/sites/default/files/microsites/ostp/PCAST/pcast_forensic_science_report_final.pdf.
Biasotti, Alfred A. 1959. “A Statistical Study of the Individual Characteristics of Fired Bullets.” Journal of Forensic Sciences 4 (1): 34–50.
Chu, Wei, Robert M Thompson, John Song, and Theodore V Vorburger. 2013. “Automatic identification of bullet signatures based on consecutive matching striae (CMS) criteria.” Forensic Science International 231 (1–3): 137–41.
Clarkson, James A, and C Raymond Adams. 1933. “On Definitions of Bounded Variation for Functions of Two Variables.” Transactions of the American Mathematical Society 35 (4). JSTOR: 824–54.
Cleveland, William S. 1979. “Robust Locally Weighted Regression and Smoothing Scatterplots.” Journal of the American Statistical Association 74 (368). Taylor & Francis, Ltd.: 829–36. http://www.jstor.org/stable/2286407.
Giannelli, Paul C. 2011. “Ballistics Evidence Under Fire.” Criminal Justice 25 (4): 50–51.
Hamby, James E., David J. Brundage, and James W. Thorpe. 2009. “The Identification of Bullets Fired from 10 Consecutively Rifled 9mm Ruger Pistol Barrels: A Research Project Involving 507 Participants from 20 Countries.” AFTE Journal 41 (2): 99–110.
Hofmann, Heike, and Eric Hare. 2016. Bulletr: Algorithms for Matching Bullet Lands.
National Research Council. 2009. Strengthening Forensic Science in the United States: A Path Forward. Washington, DC: The National Academies Press. doi:10.17226/12589.
Nichols, Ronald G. 2003. “Consecutive Matching Striations (CMS): Its Definition, Study and Application in the Discipline of Firearms and Tool Mark Identification.” AFTE Journal 35 (3): 298–306.
OpenFMC. 2014. X3pr: Read/Write Functionality for X3p Surface Metrology Format.
Sensorfar. 2017. SensoMATCH Bullet Comparison Software.
Vorburger, T.V., J.-F. Song, W. Chu, L. Ma, S.H. Bui, A. Zheng, and T.B. Renegar. 2011. “Applications of Cross-Correlation Functions.” Wear 271 (3–4): 529–33. doi:http://dx.doi.org/10.1016/j.wear.2010.03.030.
Xie, F., S. Xiao, L. Blunt, W. Zeng, and X. Jiang. 2009. “Automated Bullet-Identification System Based on Surface Topography Techniques.” Wear 266 (5–6): 518–22. doi:http://dx.doi.org/10.1016/j.wear.2008.04.081.